A data-driven approach to neural architecture search initialization

Abstract Algorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed tailor initialization techniques for NAS. However, literature shows that good initial set solutions facilitates finding optima. Therefore, this study, we propose data-driven technique initialize population-based NAS algorithm. First, perform calibrated clustering analysis space, second, extract centroids use them We benchmark our approach against random Latin hypercube sampling using three algorithms, namely genetic algorithm, an evolutionary aging evolution, on CIFAR-10. More specifically, NAS-Bench-101 leverage availability benchmarks. The results show compared sampling, enables achieving significant long-term improvements two baselines, sometimes various scenarios (various training budget). Besides, also investigate how population gathered tabular can be used improving another dataset, So2Sat LCZ-42. Our similar target despite limited budget. Moreover, analyse distributions obtained find provided by retrieving local optima (maxima) high fitness configurations.